Visualization Enabled Mixed Initiative Systems

Mixed-initiative systems (MIS) are interactive problem solvers where human and machine intelligence are combined for their respective superiority. A flexible model of interaction is employed and each problem-solving agent (human or machine) contributes to the tasks what it does best, in the best opportune task context. Several interaction models have been proposed, such as natural language dialogs, direct manipulation interfaces, and interactive intelligent agents. However, there has been little work on MIS using visual representations as an interaction medium, contrarily to how ubiquitously humans use and benefit from diagrams while solving problems. We outline in this paper the challenges facing the design of visual representations in such MIS and provide a case study in resource management tasks. Our main conclusion is that to achieve effective visual representations so that users can both easily perform tasks and perceive the results of intelligent agents, analyses of the underlying reasoning tasks must be carefully carried out to identify and map cognitive tasks to perceptual tasks, and user studies are needed to find the optimal visual rendering of information needed for perceptual tasks. In scaling the visual representation to support more complex user tasks, we observe that users switch to a different mode of problem solving by relying more on capabilities of intelligent agents. Different visualizations are thus required for the effective support of these two different task scenarios.